Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add ipex to quicktour #2122

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Conversation

jiqing-feng
Copy link
Contributor

Hi @echarlaix @IlyasMoutawwakil . As ipex reactor is done in optimum-intel, it's time to update the readme and add ipex to quicktour.

Copy link
Collaborator

@echarlaix echarlaix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jiqing-feng but the quicktour looks a bit crowded already, I don't think we want to add more code snippet for now : would make more sense to have a description explaning when to use ipex vs openvino vs neural-compressor (+ a link towards a benchmark could be very nice also) as it's currently a bit confusing for users what should be used when + what are the integrations for each. I think it also makes sense to keep all code snippet related to optimum-intel in the dedicated section to not duplicate them multiple times (same applied for others sections) we can just add links in the quickstour instead

Signed-off-by: jiqing-feng <[email protected]>
@jiqing-feng
Copy link
Contributor Author

jiqing-feng commented Dec 11, 2024

Thanks @jiqing-feng but the quicktour looks a bit crowded already, I don't think we want to add more code snippet for now : would make more sense to have a description explaning when to use ipex vs openvino vs neural-compressor (+ a link towards a benchmark could be very nice also) as it's currently a bit confusing for users what should be used when + what are the integrations for each. I think it also makes sense to keep all code snippet related to optimum-intel in the dedicated section to not duplicate them multiple times (same applied for others sections) we can just add links in the quickstour instead

Agree, do you think we should use optimum-intel pipeline instead? In this case, users can pass accelerator=ipex;ov;neural-compressor to choose which kind of backend they want to use. The descriptions should be:

To apply optimum-intel optimization, you just need to import pipeline from optimum-intel instead of transformers, and choose accelerator from ov(link), ipex(link), neural-compressor(link) to get different acceleration. Following codes:

- from transformers import pipeline
+ from optimum.intel.pipelines import pipeline

model_id = "distilbert-base-uncased-finetuned-sst-2-english"

  # Run inference!
- classifier = pipeline("text-classification", model=model_id)
+ classifier = pipeline("text-classification", model=model_id, accelerator=accelerator)
results = classifier("He's a dreadful magician.")

If we introduce optimum-intel in optimum quicktour in this way, it could be more clean and easy to understand for users.
Do you mind update the readme like this? If so, please open a new PR cause I don't have the right to delete OV context. Thanks!

@yao-matrix
Copy link

Thanks @jiqing-feng but the quicktour looks a bit crowded already, I don't think we want to add more code snippet for now : would make more sense to have a description explaning when to use ipex vs openvino vs neural-compressor (+ a link towards a benchmark could be very nice also) as it's currently a bit confusing for users what should be used when + what are the integrations for each. I think it also makes sense to keep all code snippet related to optimum-intel in the dedicated section to not duplicate them multiple times (same applied for others sections) we can just add links in the quickstour instead

@echarlaix , we're fine w/ the "add link" idea.
For positioning, do you think it helps to position ipex + inc as intel's HF HW-accelerated PyTorch-native runtime, no need for proprietary format/conversion/... ? For inc, we position it as intel's quantization lib(like autoawq etc. in ecosystem), but have intel enhancement algorithms like autoround, and we are planning to deprecate its runtime role and move runtime role back to ipex backend. So, IPEXModel can run both models quantized by ecosystem libraries(autoawq, gptqmodel etc.) and models quantized by inc.
It needs time, but it's what we will do in 2025.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants